Variational Inference in Probabilistic Programs a formal derivation of a Black-Box approach

Probabilistic models are used in many elds to tackle di erent problems, ranging from image recognition to diagnosing diseases. The advantage of using models is that we can split the encoding of our problem into a probabilistic model from the ways we solve it. We can also classify models to develop some class-speci c, but not problem-speci c algorithms to solve given tasks. These algorithms are called inference algorithms. These classes of models ease the process of solving tasks, but scientists still need to develop class-speci c algorithms. A new approach, at the intersection of Programming Languages and Machine Learning, is called Probabilistic Programming. One of the goals of this approach is to let the computer do the inference automatically, so we do not have to develop class-speci c inference algorithms to accomplish our tasks. Probabilistic Programming Languages usually extend probabilistic models. However, researchers still need to derive general inference algorithms for probabilistic programs. In this report, we give a short introduction to probabilistic models and probabilistic programming. We encode probabilistic programs into a probabilistic transition system, and transform our inference task into an optimisation problem. We simplify this optimisation problem to derive a new, general, variational inference algorithm for probabilistic programs.